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Amendments to the claims (This listing replaces all prior versions ): 

1. (cancelled). 

2. (cancelled). 

3. (cancelled). 

4. (withdrawn) A method of enabling a user to create an instance in a formal language of 
the kind which has a strictly defined syntax, comprising 

providing a graphically displayed list of entries which are expressed in natural language 
and which do not comply with said syntax, 

permitting the user to point to an entry on said list, and 

automatically generating said instance corresponding to the identified entry in the list in 
response to said pointing. 

5. (withdrawn): A method of generating a table for aiding conversion of voiced utterances 
to control commands for use in controlling an operating system of a computer to achieve desired 
actions in an application program running under the operating system, said application program 
including menus and control buttons, said method comprising 

automatically by computer parsing an application program to identify menu entries and 
control buttons, and 

automatically by computer placing a table entry in said table for each menu entry and 
control button found in the application program, each table entry placed in said table containing 
one of said control commands corresponding to said menu entry or control button. 

6. (previously presented): A voice user interface system for producing input to a 
computer, said computer having a display, said display having a pointer indicating a position on 



Applicant 
Serial No. 
Filed 
Page 



Thomas R. Firman 
09/783,725 
February 14, 2001 
3 of 14 



Attorney's Docket No.: 10591-003008 



said display, and a program for execution on said computer, a state of said program comprising a 
configuration on said display, said configuration being associated with control of said program 
and having a graphical element, the system comprising 

a voice recognizer for recognizing a voiced utterance and for providing corresponding 
signals as input to said computer, and 

a converter for converting said voiced utterance into a command string including a 
command directing motion of said pointer relative to a graphical element of said configuration. 

7. (previously presented): The system of claim 6 wherein said command string further 
comprises a command to said program. 

8. (previously presented): A voice user interface system for recognizing a voiced 
utterance and producing corresponding input to a program for execution on a computer, 
comprising 

a voice recognizer for recognizing a voiced utterance and for providing a corresponding 
signal as an input to said computer, and 

a converter for converting said voiced utterance to an output string for delivery as input 
to said computer based on an evaluation of said voiced utterance and on a state of the subsystem 
comprising said voice recognizer and said converter. 

9. (previously presented): A voice user interface system for recognizing a voiced 
utterance and producing corresponding input to a program for execution on a computer, 
comprising 

a voice recognizer for recognizing a voiced utterance and for providing a corresponding 
signal as an input to said computer, and 

a converter for converting said voiced utterance to an output string for delivery as input 
to said computer based on an evaluation of said voiced utterance and on a state of said program. 
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10. (previously presented): The system of claim 9 further comprising commands to said 
program having a format to carry associated text strings as arguments, further comprising means 
for converting a series of voiced utterances into commands with said associated text as output of 
said device. 

11. (previously presented): The system of claim 9 wherein said program, when operated 
without said converter, offers to its user, menu selections that said user selects via keyboard 
input, and wherein said converter, when used to select the same menu selection based on a 
voiced utterance, produces a series of operating system events in response to said keyboard 
input. 

12. (previously presented): The system of claim 9 wherein said operating system of said 
computer maintains an event queue, said converter delivering said output string to said event 
queue. 

13. (previously presented): A system for enabling voiced utterances to be substituted for 
manipulation of a pointing device to control motion of a displayed location indicator on a 
computer display, 

the indicator being moved by an operating system in a computer in response to control 
signals received from the pointing device, comprising 

a voice recognizer for recognizing a voiced utterance, and 

an interpreter functionally connected to said voice recognizer for converting a voiced 
utterance into control signals which will cause movement of the indicator in a desired direction 
aided by the operating system in the computer, said movement continuing unabated until stopped 
by an action of the user. 

14. (currently amended) A voice user interface device comprising 

means for converting a voiced utterance belonging to a set of voiced utterances into a 
corresponding signal as an input to a computer or into an internal command to the interface 
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device, the voic e d utteranc e belonging to a s e t of voic e d utt e ranc e s that the m e ans for conv e rting 
is configur e d to convert, the s e t b e ing a subset of all possibl e voic e d utt e ranc e s, th e int e rnal 
command b e ing a command to chang e to a differ e nt s e t th e s e t of utt e ranc e s that th e m e ans for 
conv e rting is configured to conv e rt, and 

the means for converting a voiced utterance being configured to change the set of voiced 
utterances upon receipt of the internal command, and 

means for recognizing the voiced utterance as either one to be converted to said signal or 
as one to be converted to said internal command. 

15. (previously presented) A voice user interface system for recognizing a voiced 
utterance and producing corresponding input to a program for execution on a computer, 
comprising 

a voice recognizer for recognizing said voiced utterance, 

a converter for converting said recognized voiced utterance to an output string of 
characters or commands for input to said computer, 

a set of representations, one such representation for each voiced utterance recognized by 
said voice recognizer, said representations internal to said voice recognizer and said converter, 

a set of output strings produced by said voice recognizer and said converter as input to 
said program, and 

a mapping from a member of said set of internal representations to a member of said set 
of output strings, said mapping being multiple-to-one and being used by said converter. 

16. (withdrawn) A method that is at least partially automated, comprising 

assembling a table of entries, each of the entries including an utterable token, and 
program control information to be invoked in response to uttering of the token. 
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17. (withdrawn) The method of claim 16 wherein the tokens are arranged hierarchically 
in the table. 

18. (withdrawn) The method of claim 16 wherein the tokens comprise names that are 
displayed in a user interface. 

19. (withdrawn) The method of claim 18 wherein the names comprise menu items, button 
captions, or words. 

20. (withdrawn) The method of claim 16 wherein the program control information 
comprises sequences of instructions corresponding to each of the tokens. 

21. (withdrawn) The method of claim 20 wherein the sequences of instructions are 
command strings in the form of events to be entered into an event queue of an operating system. 

22. (withdrawn) The method of claim 16 wherein the sequence of instructions 
corresponding to a given token is the sequence of instructions that is executed by the program 
when the corresponding token is invoked by the user via a user interface. 

23. (withdrawn) The method of claim 16 wherein the assembling comprises 
automatically analyzing program instructions to identify tokens for inclusion in the table. 

24. (withdrawn) The method of claim 17 wherein the analyzing comprises 
identifying user interface names that appear in the program instructions. 

25. (withdrawn) The method of claim 16 wherein the assembling comprises 

automatically analyzing the program instructions to identify program control information 
for inclusion in the table. 

26. (withdrawn) The method of claim 25 wherein the analyzing comprises 



identifying command strings associated with the tokens. 
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27. (withdrawn) The method of claim 16 wherein the assembling comprises 
automatically analyzing text to identify individual words for inclusion in the table. 

28. (withdrawn) The method of claim 16 wherein the assembling comprises recording a 
series of actions taken by a user in using a program. 

29. (withdrawn) The method of claim 16 wherein the actions include pointer operations 
and keystrokes. 

30. (withdrawn) A method that is at least partially automated, comprising 

assembling a table of entries, each of the entries including an utterable menu item or 
button caption that is displayable in a user interface, and command strings that correspond to the 
menu items and button captions, the command strings to be invoked in response to uttering of the 
items or captions. 

31. (withdrawn) A stored digital data structure comprising 

a table of entries, each of the entries including an utterable token and program control 
information to be invoked in response to uttering of the token. 

32. (withdrawn) A stored program comprising 

an instruction sequence, the execution of which assembles a table of entries each of the 
entries including an utterable token and program control information to be invoked in response to 
uttering of the token. 

33. (withdrawn) A voice controlled device comprising 
a processor, and 

a stored instruction sequence, the execution of which by the processor assembles a table 
or entries, each of the entries including an utterable token and program control information to be 
invoked in response to uttering of the token. 
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34. (withdrawn) A voice control method comprising 

executing a program having a user interface that displays invocable tokens associated by 
the program with respective instruction sequences in the program, and 

in response to an utterance corresponding to a selected one of the tokens, executing one 
of the instruction sequences that differs from the instruction sequence associated by the program 
with the selected one of the tokens. 

35. (previously presented): A method for use with a machine having a pointing device, a 
graphical user interface, and an application program, the method comprising: 

the graphical user interface being controlled at least in part by a control signal that can be 
invoked in response to the pointing device; 

the graphical user interface enabling a user to launch the application program; 

receiving a voiced utterance from a user; and 

launching the application program in response to the received voiced utterance without 
invoking the control signal. 

36. (previously presented):The method of claim 35 in which an operating system 
provides the graphical interface. 

37. (previously presented):The method of claim 35 in which the graphical user interface 
is shown on a display. 

38. (previously presented):The method of claim 35 in which the machine comprises a 
computer. 

39. (previously presented):The method of claim 35 in which there are multiple 
application programs and also comprising: 
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the graphical user interface enabling a user to launch each of the application programs, 

and 

launching at least one of the application programs in response to the received voiced 
utterance. 

40. (previously presented) The method of claim 35 in which 

there are multiple application programs, an operating system provides the graphical user 
interface, the graphical user interface is shown on a display, and the machine comprises a 
computer, 

the graphical user interface enabling a user to launch each of the application programs, 

and 

launching at least one of the application programs in response to the received voiced 
utterance. 

41 . (currently amended) A method for use with a machine having a pointing device, and a 
graphical user interface that includes a cursor and at least one other graphical item, the method 
comprising: 

receiving a voiced utterance from a user; and 

manipulating the one other graphical item , separately from the cursor, in response to 
receiving the voiced utterance. 

42. (previously presented) The method of claim 41 in which an operating system provides 
the graphical user interface. 

43. (previously presented) The method of claim 41 in which the graphical user interface 
is shown on a display. 
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44. (previously presented) The method of claim 41 in which the machine comprises a 
computer. 

45. (previously presented) The method of claim 41 in which there are multiple other 
graphical items, the method also comprising 

manipulating at least one of the other graphical items based on the voiced utterance that 
is received. 

46. (previously presented) The method of claim 45 in which the one other graphical item 
comprises a window. 

47. (previously presented) The method of claim 46 in which manipulating the one other 
graphical item comprises performing a graphical operation on the window. 

48. (previously presented) The method of claim 47 in which the graphical operation 
comprises closing the window. 

49. (previously presented) The method of claim 47 in which the graphical operation 
comprises moving the window. 

50. (previously presented) The method of claim 47 in which the graphical operation 
comprises zooming the window. 

51. (previously presented) The method of claim 47 in which the graphical operation 
comprises moving the window toward the front or rear of a stack. 

52. (previously presented) A method for use with a machine having a pointing device and 
an operating system providing a graphical user interface that includes a menu that may be 
invoked in response to the pointing device, the pointing device being capable of sending a 
control signal to the graphical user interface, the menu including selectable menu items 
associated with respective functions that may be performed, the method comprising: 

receiving a first voiced utterance from a user; and 
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displaying the menu in response to receiving the first voiced utterance without invoking 
the control signal. 

53. (previously presented) The method of claim 52 in which there are multiple menus that 
may be invoked in response to the pointing device, each of the menus including selectable menu 
items associated with respective functions that may be performed. 

54. (previously presented) The method of claim 52 in which the graphical user interface 
is shown on a display. 

55. (previously presented) The method of claim 52 in which the machine comprises a 
computer. 

56. (previously presented) The method of claim 52 also including receiving a second 
voiced utterance from a user; 

in response to receiving the second voiced utterance, performing a function associated 
with a menu item included on said displayed menu. 

57. (previously presented) The method of claim 56, also comprising, in response to the 
second voiced utterance, modifying the appearance of the included menu item. 

58. (previously presented) The device of claim 14 in which the set of voiced utterances 
the device is configured to convert is adjusted by adding an additional voiced utterance to the set. 

59. (currently amended) A voice user interface device comprising 

means for converting a voiced utterance into a corresponding signal as an input to a 
computer, 

means for converting a voiced utterance into a corresponding internal command to the 
voice user interface device to cause the voice user interface device to acc e pt information about 
change (a) which voiced utterances belong to the set of voiced utterances the device is capable of 
recognizing, the set of voiced utterances being a subset of all possible voiced utterances, and (b) 
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the set of signals and internal commands that correspond to each member of the set of voiced 
utterances the voice user interface device is capable of recognizing; and 



means for recognizing a voiced utterance as either one to be converted to a signal as an 
input to a computer or as one to be converted to an internal command. 



